In recent years, there has been an increase in demand for using camera sensors in automotive applications. For example, cameras mounted in the rear view mirror of a vehicle are used to monitor a scene in the front of the vehicle to detect possible collisions with pedestrians or other vehicles. Night vision systems have also been introduced to enhance pedestrian safety in the presence of vehicluar traffic. One such system is a pedestrian detection system that uses a monocular infra-red camera.
In such applications, it is important to estimate the motions of the camera, which, since the camera is usually mounted rigidly to the vehicle, provides measurements of the motion of the vehicle itself. The types of vehicle motion, which can be expressed as rotation and translation, are related to the pose of the camera. Therefore, it is desirable to obtain the pose of a vehicle with respect to a ground plane, i.e., the road. The vehicle pose with respect to ground plane can be used to measure the distance to targets on the road when no alternative depth estimation method is available. For example, to identify threat situations in collision avoidance systems, it is desired to have an estimate of distance to targets in the path of a vehicle so that possible collisions can be identified and predicted. Secondly, egomotion of a vehicle can be used to directly estimate the ground plane from which a region of interest for further processing can be identified. In addition, vehicle motion information can be used to track targets (e.g., other vehicles and obstacles) robustly, especially at night.
One of the most commonly used approaches for camera pose estimation is to exploit correspondences between different views and compute a relative camera pose from matching features. The camera pose can be obtained by solving equations based on epipolar constraints. For example, in D. Nister, O. Naroditsky, and J. Bergen, “Visual Odometry,” Proc. IEEE Conf. on Comp. Vision and Patt. Recog., 01:652-659, 2004, Nister et. al. proposed a visual odometry that computes camera pose from Ransac based feature matching combined with a five-point algorithm for efficient pose computation. Optical-flow based methods have been proposed that computes pose by exploiting image flow on the ground or from image alignment. A direct method based approach has been adopted to avoid explicit calculation of optical-flow and feature matching. In J. G. Stein, O. Mano, and A. Shashua, “A robust method for computing vehicle ego-motion,” 2000 and J. G. Stein and A. Shashua, “Model-based brightness constraints: On direct estimation of structure and motion,” IEEE Trans. Pattern Anal. Machine Intell., 22(9):992-1015, 2000, Stein et. al. presented a system that computes egomotion of vehicle based on points on ground plane under a constrained camera motion assumption. In this method, a set of uniformly tessellated image patches from the ground plane is selected to probabilistically integrate to solve a motion equation based on planar motion constraint and brightness constancy.
All of the above mentioned approaches are based on either strong feature matching or require sufficient texture for image flow computations. However, these conditions cannot be met easily in the infra-red domain. This is because in infra-red images taken of road scenes, the thermal response of the scenes vary smoothly in a relatively low dynamic range, which results in smoothed, low resolution images compared with visual range image data. Thus, conventional approaches cannot be directly applicable to robust pose estimation.
There are other diffulties that are encountered with the use of correspondence-based pose estimation. First, there are few good features to track in an IR images of the road scenes. In general, features of the road structure such as lane markers are not available. Secondly, good features are typically found at the periphery of image and cannot be tracked robustly due to large image motions. That is, features tracked from nearby road structures are more distinctive to match than ones farther away and provide better resolution for numerically stable solutions. However, nearby road structures move faster in the image and quickly disappear before matching can be attained.
Accordingly, what would be desirable, but has not yet been provided, are an accurate method and resulting system for estimating camera motion and hence vehicle pose and motion from a camera mounted on a vehicle that uses infra-red images for night time applications.